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DETAILED ACTION 
Response to Amendment 

1 . In response to the office action from 5/3 1/2005, the applicant has submitted a request for 
continued examination, filed 8/31/2005, amending the claims 3, 5, 7, 9-13, and 15-16 while 
adding claims 17-18 and arguing to traverse the art rejection based on the limitation regarding 
reaction time range limits used for determining that a user interaction with a speech recognition 
engine is being smoothly conducted (Amendment, pages 13-14). The appilicant's arguments have 
been fully considered but are moot with respect to the new grounds of rejection in view of 
Daudelin (U.S. Patent: 4, 797,910) and Davis et al (U.S. Patent: 5,583,922). 

Response to Arguments 

2. With respect to Claims 5 and 15-16, the applicant argues that the prior art of record fails 
to teach a progressive operator interaction (Amendment, Pages 15-16). The examiner points out 
that Davis et al (U.S. Patent: 5,583,922) teaches such a progressive interaction in the form of a 
standard update (involvement) of a user entry form at a operator station (Col. 9, Lines 1-11), 
direct interaction (parallel input of data) through data (Col. 1 1, Lines 36-58), and when necessary 
switching control to an operator for voice communication (Col. 1 1, Lines 36-58). The examiner 
also notes that Bohacek et al (U.S. Patent: 6,41 1,687) teaches a progressive interaction from a 
monitored speech recognition system session to operator involvement through switching (Col. 3, 
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Line 60- Col. 4, Line 8). Although Bohacek fails to specifically recite an intermediate parallel 
data input mode, Davis discloses such a mode, as noted above, and thus, the combination of 
Bohacek and Davis also teach the claimed progressive operator interaction. 

Thus, since Davis and the combination of Bohacek and Davis each teach a progressive 
interaction in the form of user data entry form updates, direct interactions, and direct voice 
interaction, Claims 5 and 15-16 remain rejected. 

With respect to Claims 9 and 10, in response to applicant's argument that the references 
fail to show certain features of applicant's invention, it is noted that the features upon which 
applicant relies (i.e., transferring a call to an operator when a response reaction time is exceeded, 
amendment, page 15) are not recited in the rejected claim(s). Although the claims are interpreted 
in light of the specification, limitations from the specification are not read into the claims. See In 
re Van Geuns, 988 F.2d 1 181, 26 USPQ2d 1057 (Fed. Cir. 1993). 

The applicant's arguments directed towards the remaining rejected claims have been fully 
considered but are moot with respect to the new grounds of rejection in view of Daudelin (U.S. 
Patent: 4, 797,910) and Davis et al (U.S. Patent: 5,583,922). 

The remaining dependent claims are argued as further limiting rejected independent 
claims (Amendment, Page 17), and thus, also remain rejected. 

Claim Rejections - 35 USC §103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 
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(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

4. Claims 3, 6, and 8 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Bohacek et al (U.S. Patent: 6,41 1,687) in view of Daudelin (U.S. Patent: 4, 797,910) and further 
in view of Davis et al (U.S. Patent: 5,583,922). 

With respect to Claim 3, Bohacek discloses: 

A voice information input part for inputting user's voice information from a user terminal 
(user phone call to an interactive voice response system which would inherently require a 
telephone or similar communication device, Col. 3, Lines 60-65); 

A voice recognition part for conducting voice recognition with respect to the voice 
information, and analyzing contents of the voice information (speech recognizer used to collect 
information such as account number or billing data, Col. 3 f Line 48- Col. 4, Line 8); 

A voice information mediation part for controlling a transmission path of the voice 
information in accordance with the contents of the voice information (detecting user mood based 
upon speech samples to transfer an IVR system user to an operator for assistance, Col 4, Lines 
40-48 \ and Col. 3, Lines 17-30) ; 

An interaction engine for extracting contents o.f a response corresponding to the voice 
information by referring to a knowledge database, and creating a synthesized voice in accordance 
with the contents of a response (interactive voice response unit that prompts a user for 
information, Col. 3, Lines 60-65, which would inherently require a memory for storing voice 
prompts); and 
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A voice information output part for outputting the synthesized voice, wherein the voice 
information mediation part monitors at all times whether or not the user's interaction is being 
smoothly conducted, and in a case of determining that the user's interaction is not being 
smoothly conducted, allows a third-party user to participate (requesting user information, Col 3, 
Lines 60-65, and transferring the call of a frustrated IVR system user to an operator, Col. 3, 
Lines 17-30, and Col 4, Lines 40-48). 

Bohacek does not specifically suggest the ability to transfer a call to an operator when a 
user fails to produce a response within response reaction time interval, however, Daudelin 
recites: 

The voice information mediation part determines whether or not the user's interaction is 
being smoothly conducted based on an average reaction time from a response of the interaction 
engine to a reaction of the user, and in a case where the average reaction time exceeds a first 
threshold value or in a case where the average reaction time is below a second threshold value, 
allows a third-party user to participate in the interaction between the user and the interaction 
engine from another terminal as a helper (timeout interval used to initialize a call transfer to an 
operator, which would inherently have starting and ending boundaries (lower and higher limits), 
Col 11, Lines 23-54). 

Bohacek and Daudelin are analogous art because they are from a similar field of 
endeavor in speech recognition systems utilizing an option for operator assistance. Thus, it 
would have been obvious to a person of ordinary skill in the art, at the time of invention, to 
modify the teachings of Bohacek with the use of a timeout interval to initialize a call transfer to 
an operator as taught by Daudelin in order to provide a means for determining whether an 



Application/Control Number: 09/940,522 Page 6 

Art Unit: 2655 

operator's assistance is necessary in interacting with a voice processing system that also 
minimizes substantial operator involvement (Daudelin, Col 2, Lines 52-61). 

Bohacek in view of Daudelin does not specifically suggest that an operator helps in an 
interaction between a first user and an interaction engine, however Davis teaches such an 
operator assistance method (Col. 11, Line 36- Col 12, Line 20). 

Bohacek, Daudelin, and Davis are analogous art because they are from a similar field of 
endeavor in speech recognition systems utilizing an option for operator assistance. Thus, it 
would have been obvious to a person of ordinary skill in the art, at the time of invention, to 
modify the teachings of Bohacek in view of Daudelin with the method for operator assistance 
taught by Davis in order to provide a means for screening and categorizing calls for appropriate 
operator assistance (Davis, Col 12, Line 2-5). 

With respect to Claim 6, Bohacek discloses: 

The interaction engine further includes an interaction history information storage part for 
recording interaction history on a user basis, and a helper selection part for selecting the third- 
party user that is considered to be most familiar with the contents of the interaction from the 
interaction history as a helper, and the helper most appropriate for contents of the voice 
information is selected (previous customer data used to route a call to the most appropriate 
operator, Col 4, Lines 24-39, and Fig, 1). 

With respect to Claim 8, Bohacek recites: 

Interaction history display part for displaying the interaction history stored in the 
interaction history information storage part to a third-party helper user, and a helper instruction 
part for receiving a help instruction from the third-party helper user, wherein when the help 
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instruction part receives the help instruction from the third-party helper user, the voice 
information mediation part enables the interaction between the third-party helper user and the 
user to be conducted, and when a degree of help of the third-party helper user exceeds a 
predetermined threshold value in interaction between the third-party helper user and the user, the 
interaction engine interacts only with the third-party helper user (transferring interaction control 
to an appropriate operator when an annoyance threshold is exceeded, displaying customer data, 
and previous customer data, Col 3, Lines 17-30, and Col 4, Lines 30-39), 

5. Claim 5, 11, 13, and 15-16 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Bohacek et al in view of Marx et al (U.S. Patent: 6,173,266), and further in view of Davis et al 
(U.S. Patent: 5,583,922). 

With respect to Claims 5 and 15, Bohacek discloses: 

A voice information input part for inputting user's voice information from a user terminal 
(user phone call to an interactive voice response system which would inherently require a 
telephone or similar communication device, Col 3, Lines 60-65); 

A voice recognition part for conducting voice recognition with respect to the voice 
information, and analyzing contents of the voice information (speech recognizer used to collect 
information such as account number or billing data, Col 3, Line 48- Col 4, Line 8)\ 

A voice information mediation part for controlling a transmission path of the voice 
information in accordance with the contents of the voice information (detecting user mood based 
upon speech samples to transfer an IVR system user to an operator for assistance, Col 4, Lines 
40-48, and Col 3, Lines 17-30) ; 
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An interaction engine for extracting contents of a response corresponding to the voice 
information by referring to a knowledge database, and creating a synthesized voice in accordance 
with the contents of a response (interactive voice response unit that prompts a user for 
information, Col 3, Lines 60-65, which would inherently require a memory for storing voice 
prompts) ', and 

A voice information output part for outputting the synthesized voice, wherein the voice 
information mediation part monitors at all times whether or not the user's interaction is being 
smoothly conducted, and in a case of determining that the user's interaction is not being 
smoothly conducted, allows a third-party user to participate in interaction between the user and 
the interaction engine from another terminal as a helper (requesting user information, Col 3, 
Lines 60-65, and transferring the call of a frustrated IVR system user to an operator, Col 3, 
Lines 17-30, and Col 4, Lines 40-48). 

Bohacek does not specifically suggest the ability to transfer a call to an operator when a 
response reaction time limit is exceeded, however, Marx recites: 

The voice information mediation part determines whether or not the user's interaction is 
being smoothly conducted based on an average reaction time from a response of the interaction 
engine to a reaction of the user, and in a case where the average reaction time exceeds a first 
threshold value or in a case where the average reaction time is below a second threshold value, 
allows a third-party user to participate in the interaction between the user and the interaction 
engine from another terminal as a helper (timeout condition used to initialize a call transfer to an 
operator, Col 8, Lines 20-31, and Col 9, Lines 52-65). 
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Bohacek and Marx are analogous art because they are from a similar field of endeavor in 
interactive voice response systems. Thus, it would have been obvious to a person of ordinary 
skill in the art, at the time of invention, to combine the use of a timeout condition to initialize a 
call transfer to an operator as taught by Marx with the interactive voice response system capable 
of transferring a caller to an operator upon detecting a predetermined level of frustration based 
on speech data as taught by Bohacek in order to implement a fallback method when speech is not 
properly recognized (Marx, Col 9, Line 66- Col 10, Line 2). 

Bohacek in view of Marx does not specifically suggest the ability to determine 
interaction progress through interaction time and access amounts in order to initialize parallel 
input operator/user interaction, however Davis recites: 

The contents of interaction with the user is displayed to the third-party user and the 
contents of interaction is updated by the third-party user, parallel input in which the third-party 
user conducts an input in parallel with the user, to switching in which the third-party user 
directly interacts with the user (user information transferred to an operator and direct 
communication through data and voice, Col 11, Line 36- Col 12, Line 20; Col 8, Lines 20-31; 
and Col 9, Lines 1-33). 

Bohacek, Marx, and Davis are analogous art because they are from a similar field of 
endeavor in interactive voice response systems. Thus, it would have been obvious to a person of 
ordinary skill in the art, at the time of invention, to combine the use of a parallel user/operator 
input as taught by Davis with the interactive voice response system and method capable of 
transferring a caller to an operator based upon a frustration level, recognition engine error 
amount, or a timeout condition as taught by Bohacek in view of Marx in order to provide a 
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means for screening and categorizing calls for appropriate operator assistance (Davis, Col 12, 
Line 2-5). 

Claim 11 contains subject matter similar to Claim 6, and thus, is rejected for the same 
reasons. 

Claim 13 contains subject matter similar to Claim 8, and thus, is rejected for the same 
reasons. 

Claim 16 contains subject matter similar to Claims 5 and 10, and thus, is rejected for the 
same reasons. 

6. Claims 9-10 are rejected under 35 U.S.C. 103(a) as being unpatentable over Bohacek et 
al (U.S. Patent: 6,411,687) in view of Davis et al (U. S. Patent: 5,583,922). 
With respect to Claim 9, Bohacek discloses: 

A voice information input part for inputting user's voice information from a user terminal 
(user phone call to an interactive voice response system which would inherently require a 
telephone or similar communication device, Col 3, Lines 60-65); 

A voice recognition part for conducting voice recognition with respect to the voice 
information, and analyzing contents of the voice information (speech recognizer used to collect 
information such as account number or billing data, Col 3 t Line 48- Col 4, Line 8); 

A voice information mediation part for controlling a transmission path of the voice 
information in accordance with the contents of the voice information (detecting user mood based 
upon speech samples to transfer an IVR system user to an operator for assistance, Col 4, Lines 
40-48, and Col 5, Lines 17-30); 
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An interaction engine for extracting contents of a response corresponding to the voice 
information by referring to a knowledge database, and creating a synthesized voice in accordance 
with the contents of a response (interactive voice response unit that prompts a user for 
information, Col 3, Lines 60-65, which would inherently require a memory for storing voice 
prompts); and 

A voice information output part for outputting the synthesized voice, wherein the voice 
information mediation part monitors at all times whether or not the user's interaction is being 
smoothly conducted, and in a case of determining that the user's interaction is not being 
smoothly conducted, allows a third-party user to participate (requesting user information, Col 3, 
Lines 60-65, and transferring the call of a frustrated IVR system user to an operator, Col 3, 
Lines 17-30, and Col 4, Lines 40-48). 

Bohacek does not specifically suggest that an operator helps in an interaction between a 
first user and an interaction engine, however Davis teaches such an operator assistance method 
(Col 11, Line 36- Col 12, Line 20). 

Bohacek and Davis are analogous art because they are from a similar field of endeavor in 
speech recognition systems utilizing an option for operator assistance. Thus, it would have been 
obvious to a person of ordinary skill in the art, at the time of invention, to modify the teachings 
of Bohacek with the method for operator assistance taught by Davis in order to provide a means 
for screening and categorizing calls for appropriate operator assistance (Davis, Col 12, Line 2- 

With respect to Claim 10, Bohacek in view of SM^wer teaches the interactive voice 
response system and method capable of transferring a caller to an operator upon detecting a 
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predetermined level of frustration based on speech data, as applied to Claim 9. Although 
Bohacek in view of SSKgter does not specifically suggest method storage as a program on a 
computer readable medium, <8k9EBB0H^CR&40HSff9^K4^it would have been obvious to 
one of ordinary skill in the art, at the time of invention, to store the IVR method taught by 
Bohacek in view of SSS&ti? as a program on a computer readable medium to increase method 

A 

compatibility and usability by providing a means for method use with multiple computer 

systems. 

Allowable Subject Matter 

7. Claims 7, 12, 14, 17, and 18 are allowed. 

8. The following is a statement of reasons for the indication of allowable subject matter: 
With respect to Claims 7, 17, and 18, the prior art of record does not explicitly teach nor 

fairly suggests: the ability of an operator to assume complete control of an voice interaction 
engine in an interactive voice response system when it is detected that only an operator's voice 
continues for a predetermined time period during an error-initiated operator interaction session, 
in combination with a help request notification used for informing an operator of a user's failure 
to respond to an interactive voice system (error) and allowing the operator to voluntarily interact 
with the user in response to such a notification. Further the prior art of record also does not teach 
or fairly suggest the above features in combination with the aforementioned voice response 
system, wherein the voice response system utilizes a voice information mediation portion for 
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performing the interaction error detection based on response time and sound quality thresholds to 
decide whether to transfer a user to third party operator assistance. 

Claims 12 and 14 further limit allowable claim 7, and thus are also considered allowable. 

Conclusion 

9. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure: 

Scherer (U.S. Patent: 5,867,562)- teaches a method for call screening that allows an 
operator to aid in the use of an IVR system. 

10. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to James S. Wozniak whose telephone number is (571) 272-7632. 
The examiner can normally be reached on M-Th, 7:30-5:00, F, 7:30-4, Off Alternate Fridays. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Wayne Young can be reached on (571) 272-7582. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



James S. Wozniak 
10/12/2005 
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